Multimedia applications handle enormous and various information, and it has been necessary to rapidly process such information for the practical multimedia applications. A data compression/decompression technology is indispensable for rapidly processing the information.
One example of such a data compression/decompression technology is an “MPEG (Moving Picture Expert Group)” system. The MPEG system is standardized by an MPEG committee (ISO/IEC JTC1/SC29/WG11) affiliated with ISO (International Organization for Standardization)/IEC (International Electro-technical Commission).
The MPEG consists of three sections. Part 1, “MPEG systems part,” (ISO/IEC IS 11172 Part 1:Systems) defines multiplex structures and synchronous systems of video data and audio data. Part 2, “MPEG video part” (ISO/IEC IS 11172 Part 2:Video) defines a high efficiency coding system and a format of video data. Part 3, “MPEG audio part” (ISO/IEC IS 11172 Part 3:Audio) defines high efficiency coding system and a format of audio data.
The video data handled in the MPEG video part involves a moving image, which is constituted by dozens of (e.g., 30) frames (static images, segments) per second. The video data is constituted of a six-layer hierarchical structure, which includes a Sequence, a GOP (Group Of Pictures), a Picture, a Slice, a Macroblock and a Block. The number of Slices forming one Picture may vary, and the number of Macroblocks forming a Slice may also vary.
Further, the MPEG standard defines two systems, MPEG-1 and MPEG-2, mainly depending on a difference in encoding rates. In MPEG-1, a frame corresponds to a Picture. In MPEG-2, a frame or a field may correspond to a Picture. One frame includes two fields.
It is noted that a structure in which the frame corresponds to a Picture is referred to as a frame structure, whereas a structure in which the field corresponds to a Picture is referred to as a field structure.
The MPEG uses a compression technology called an interframe prediction. In the interframe prediction, data is compressed based on a time correlation between frames. A bidirectional prediction is performed in the interframe prediction. The bidirectional prediction is a prediction that uses a forward prediction predicting a present reconstructed image from a past reconstructed image (or Picture), together with a backward prediction predicting a present reconstructed image from a future reconstructed image.
The bidirectional prediction defines three types of Pictures referred to as an I-Picture (Intra-Picture), P-Picture (Predictive-Picture) and B-Picture (Bidirectionally predictive Picture).
The I-Picture is generated independent of the past or future reconstructed image. For performing a random access, at least one I-Picture is required in the GOP. All Macroblock Type within I-Picture are IntraFrame prediction images.
P-Picture is generated by the forward prediction (the prediction from a past I-Picture or P-Picture). A Macroblock Type within P-Picture includes both IntraFrame prediction image and Forward Inter Frame prediction image.
B-Picture is generated by the bidirectional prediction. In the bidirectional prediction, B-Picture is generated by any one of the three predictions below.                Forward prediction; a prediction from a past I-Picture or B-Picture        Backward prediction; a prediction from a future I-Picture or P-Picture        Bidirectional prediction; a prediction from past and future I-Picture or P-Picture        
The Macroblock within B-Picture includes four types, that is, Intraframe prediction image, Forward Inter Frame prediction image, Backward Inter Frame prediction image, and Interpolative Inter Frame prediction image.
These I, P and B Pictures are respectively encoded. That is, an I-Picture is generated without a past or future Picture. On the other hand, a P-Picture is generated only when there is a past Picture, and a B-Picture is generated only when there is a future Picture.
However, when the Mactoblock Type is Interpolative Inter Frame, Mactoblocks of P-Picture and B-Picture could be generated even if there is no past or future Picture.
In Inter Frame prediction, initially, an I-Picture is periodically generated. Subsequently, a frame that is some frames ahead of the I-Picture is generated as a P-Picture. This P-Picture is generated by a prediction in one-way (in the forward direction) from past to present. Thereafter, a frame located before the I-Picture and after the P-Picture is generated as a B-Picture. When the B-Picture is generated, a best prediction method is selected from the three predictions, the forward prediction, the backward prediction and the bidirectional prediction. Generally, in a continuous moving image, a present image is similar to preceding/succeeding images, except for only a part. It is assumed that a frame (e.g., I-Picture) and the succeeding frame (e.g., P-Picture) are almost the same, and if there is a difference between these frames, only the difference (data of B-Picture) is extracted for compression. This allows the compression to be performed in accordance with time correlation of data between the frames.
A data stream (bit stream) of video data encoded based on the MPEG video part, as described above, is referred to as an MPEG video stream (herein after referred to as a video stream).
It is noted that MPEG-1 mainly corresponds to storage media such as a video CD (Compact Disc) and a CD-ROM (CD-Read Only Memory). MPEG-2 corresponds, not only to the storage media such as a video CD, a CD-ROM, a DVD (Digital Video Disc), a video tape and a memory card using a nonvolatile semiconductor memory, but also to all transmission media including communication media such as LAN (Local Area Network) and broadcasting media such as terrestrial broadcasting, satellite broadcasting and CATV (Community Antenna Television).
Motion Compensated prediction (MC) and Discrete Cosine Transform ((DCT) are core technologies used in the MPEG video part. A coding technology using both MC and DCT are referred to as a hybrid coding technology. ADCT code is used in the MPEG video part at the time of encoding, to decompose an image (a video signal) into frequency components for processing. Inverse DCT (IDCT) is used at the time of decoding to restore the frequency components to an image.
FIG. 4 is a block circuit diagram of a conventional MPEG video decoder 101.
MPEG video decoder 101 includes a control core circuit 102, a bit buffer 103, a frame buffer 104, an MPEG decode core circuit 105, a data bus 106, a display circuit 107, a memory controller 108 and buffers 109-115 with FIFO (First-In-First-Out) configurations. Circuits 102-115 constituting MPEG video decoder 101 are mounted on one LSI chip.
Control core circuit 102 controls circuits 103-115.
A video stream transferred from a transmission medium 130 is initially input to buffer 109, transferred therefrom in the order of data bus 106, memory controller 108 and to bit buffer 103, and written into bit buffer 103. Transmission medium 130 includes a storage medium (i.e., a video CD, a CD-ROM, a DVD, a video tape, a memory card or the like), a communication medium (LAN or the like), and a broadcasting medium (terrestrial broadcasting, satellite broadcasting, CATV or the like).
Bit buffer 103 is constituted of a ring buffer including a SDRAM (Synchronous Dynamic Random Access Memory) with FIFO configuration, and successively stores video streams transmitted from transmission medium 130.
Bit buffer 103 is provided because each of the I-Picture, the P-Picture and the B-Picture has a different data size. The data size of the I-Picture is approximately 30 kilo byte, the data size of the P-Picture is approximately 10-15 kilo byte, and the data size of the B-Picture is approximately 0-6 kilo byte. However, bit rate of the video stream transmitted from transmission medium 130 is constant. MPEG decode core circuit 105 performs a process for each Picture, and the process time is different depending on the data size of each Picture. This causes a problem that some Pictures may not be processed in MPEG decode core 105 if the video stream transmitted from transmission medium 130 is directly transferred to MPEG decode core circuit 105. To prevent such a problem, bit buffer 103 is provided as a buffer memory for the video stream transferred from transmission medium 130, absorbing a difference between data sizes of I-, P- and B-Pictures.
Frame buffer 104 includes an SDRAM, and its inside is divided into three regions (i.e., a forward reference area 120, a rearward reference area 121 and a B-Picture storage area 122).
It is noted that bit buffer 103 and frame buffer 104 are provided in different regions in one SDRAM, in order to reduce the number and cost of components in MPEG video decoder 101.
Input/output data bus width (bit width) of the SDAM, in which bit buffer 103 and frame buffer 104 are provided, is set to 32 bit. Thus, input/output data bus width of memory controller 108 and data bus width of data bus 106 are also set to 32 bit.
Memory controller 108 controls reading and writing operations of bit buffer 103 and frame buffer 104.
The video stream stored in bit buffer 103 is read, by memory controller 108, for each Picture per one frame period, and the read video stream is transferred from memory controller 108 through data bus 106 and buffer 110 in this order, and input into MPEG decode core circuit 105.
For the video stream of one Picture that has been input, MPEG decode core circuit 105 first performs a variable length decoding process based on the Huffman code. Next, a dequantization process based on a quantization threshold is performed for the result of the variable length decoding process, to obtain a DCT (Discrete Cosine Transform) coefficient. Thereafter, the obtained DCT coefficient is subjected to an IDCT process, and finally an MC (Motion Compensated prediction) process is performed for the result of IDCT (Inverse DCT) process.
The result of the MC process by MPEG decode core circuit 105 is transferred through buffer 114, data bus 106, memory controller 108 and to frame buffer 104 in this order, and stored in any of the areas 120-122 of frame buffer 104 by memory controller 108.
Further, the data read from each of the areas 120-122 of frame buffer 104 by memory controller 108 is transferred in the order from memory controller 108, data bus 106 and to any one of the buffers 111-113, and is input into MPEG decode core circuit 105. Here, the data read from forward reference area 120 is transferred via buffer 111, the data read from rearward reference area 121 is transferred via buffer 112, and the data read from B-Picture storage area 122 is transferred via buffer 113.
A future I-Picture or P-Picture, used when a backward prediction is performed in the MC process by MPEG decode core circuit 105, is stored in forward reference area 120. A past I-Picture or P-Picture, used when a forward prediction is performed in the MC process, is stored in rearward reference area 121. A B-Picture is stored in B-Picture storage area 122.
I-Picture or P-Picture stored in forward reference area 120 and rearward reference area 121 are used as base data for performing the forward or backward prediction, so that they must be kept stored in the areas 120 and 121. B-Picture stored in B-Picture storage area 122 is not handled as base data, and hence the B-Picture will be unnecessary once it is output to the outside of MPEG video decoder 101. It is noted that areas 120-122 are also referred to as planes.
The Picture data read by memory controller 108 from any one of areas 120-122 of frame buffer 104 is transferred from memory controller 108 through data bus 106 and buffer 115 in this order, and is input into display circuit 107.
Display circuit 107 generates a video signal (an image signal) from the data of the Picture, and outputs the video signal to an external device 131 connected to MPEG video decoder 101. For example, if a display is connected as external device 131, the display displays the video signal as an image. If a storage medium (a video tape, a memory card and so forth) is connected as external device 131, the video signal is stored in the storage medium.
MPEG video decoder 101 configured as described above is incorporated into a movie camera, a still camera, a television, a video CD reproducing device, a DVD reproducing device and so forth. When MPEG video decoder 101 is incorporated into a movie camera or a still camera, transmission medium 130 is replaced by an image pick-up device, such as a CCD (Charge Coupled Device), and the signal processing circuit therefor.
FIG. 5 schematically shows a storing state of luminance (Y) data and color difference (C) data stored in forward reference area 120 and rearward reference area 121 of frame buffer 104 including an SDRAM.
Forward reference region 120A is provided with storage area 140 for forward-reference luminance data yf and a storage area 141 for forward-reference color-difference data cf. Further, rearward reference area 121 is provided with a storage area 142 for rearward-reference luminance data yr and a storage area 143 for rearward-reference color-difference data cr.
The data size of the color-difference data is approximately half the data size of the luminance data. The data size of each storage area 141, 143 are set to approximately half the data size of each storage area 140, 142.
It is noted that provision of storage areas 140-143 in areas 120 and 121 of frame buffer 104, for respectively storing data yf, cf, yr and cr therein as described above, is referred to as “memory mapping.”
Generally, upon accessing the SDRAM, it is required to set a predetermined command that is determined by a cast sequence and a burst range defined by the SDRAM. Such setting of the predetermined command is referred to as “command overhead.” Thus, in a period from a time point at which an access starts to a time point at which actual data writing or reading starts, delay time for a time period required for the command overhead occurs. It is impossible to make this time period required for the command overhead (delay time) equal to or shorter than 6 to 7 operation clocks of the SDRAM.
FIG. 6 schematically shows the order of the memory access in a case where data for one Macroblock is read from areas 120 and 121 of frame buffer 104 including an SDRAM with an input/output data bus width of 32 bit.
The memory access is performed in the order as follows: command overhead corn for storage area 140; reading of forward-reference luminance data yf from storage area 140; command overhead com for storage area 141; reading of forward-reference color-difference data cf from storage area 141; command overhead corn for storage area 142; reading of rearward-reference luminance data yr from storage area 142; command overhead corn for storage area 143; and reading of rearward-reference color-difference data cr from storage area 143.
Therefore, time period Ti required for the memory access in such a case can be obtained by the equation (1) below.T1=4×t1+2×t2+2×t3  (1), wherein
t1 is a time period required for command overhead com,
t2 is a time period required for reading of luminance data yf, yr, and
t3 is a time period required for reading of color difference data cf, cr.
In the MC process by MPEG decode core circuit 105, for restoring luminance data for half the one Macroblock (half-Macroblock), the luminance data for half-Macroblock may be read from forward reference area 120 or rearward reference area 121.
As shown in FIG. 7, the luminance data that must be read from forward reference area 120 or rearward reference area 121 to restore the luminance data for the half-Macroblock is data of 9 pixels (picture element)×17 pixels (picture element).
Generally, a minimum unit that can access the SDRAM will be equal to the value of the burst range defined by the SDRAM multiplied by the input/output data bus width.
Because the minimum value of the burst range is “2,” the minimum unit for accessing the SDRAM having the input/output data bus width of 32 bit will be 2×32 bit. It is noted that data size of one pixel picture element) is 8 bit. Therefore, the minimum unit for accessing the SDRAM with the input/output data bus width of 32 bit will be horizontally arranged 8 pixels.
Thus, as shown in FIG. 7B, it is necessary to read 9 pixels×24 pixels of data in order to read the luminance data of the half-Macroblock (9 pixels×17 pixels of data) from frame buffer 104 including the SDRAM with the input/output data bus width of 32 bit. Thus, data of 24 pixels, i.e., three times the horizontal 8 pixels, must be read in order to read data of horizontal 17 pixels, since the minimum unit for accessing the SDRAM with the input/output data bus width of 32 bit is horizontal 8 pixels. Except for the necessary data of 9 pixels×17 pixels of the read data of 9 pixels×24 pixels, the remaining data of 9 pixels×7 pixels will be unnecessary.
In recent years, faster operation speed of MPEG video decoder 101 is required. For increasing the operation speed, there are a method of speeding up frame buffer 104 and data bus 106 to increase the operation frequency, and a method of further expanding the input/output data bus width (bit width) to be wider than 32 bit (e.g., 48 bit, 64 bit, 128 bit and so forth). However, frame buffer 104 with high operation frequency will have an increased power consumption, in addition to being expensive. (For example, Rambus DRAM faster than the SDRAM will be more expensive than the SDRAM.) Further expansion of input/output data bus width increases the number of terminals in an LSI as well as the number of chips in the SDRAM included in frame buffer 104, causing increase of a substrate mounting area of MPEG video decoder 101 and higher cost. Thus, it has been required to increase the operation speed of MPEG video decoder 101 without making the operation frequency of frame buffer 104 higher or expanding the input/output data bus width.